Association of publication record and independent NIH funding

Background Publications may be a modifiable factor toward research project grant (RPG) funding decisions, the objective was to determine the association of publication record with later RPG receipt. Methods This was a retrospective cohort study of recipients of K01, K08, or K23 US career development awards (CDAs) starting from 2000–2015. Exposures were CDA awardees’ first-, middle-, and last-author publication counts, and the quartile of awardees’ highest and mean publication impact factors. The independent association of each exposure with time to RPG (R01 or equivalent) was determined using a Cox model, after adjustment for CDA type, awardee change in institution, and institutional CDA count. The proportion of CDA recipients with later independent funding was also determined by publication count. Results Among 6744 CDA awardees, 3943 obtained an RPG. The median time to RPG was 5.6 years (interquartile range 4.2–7.5). The number of first-authorships was associated with a shorter time to RPG (1–4 versus 0: hazard ratio [HR] 1.22, 95% confidence interval [CI] 1.10–1.36; 5–9: 1.59, 95% CI 1.40–1.79; 10–24: 1.78, 95% CI 1.54–2.07; 25+: 2.40, 95% CI 1.61–3.56). Last-authorships were associated with a shorter time to RPG (1–4 versus 0: HR 1.99, 95% CI 1.83–2.16; 5–9: 2.72, 95% CI 2.45–3.03; 10–24: 3.17, 95% CI 2.78–3.62; 25+: 3.12, 95% CI 2.17–4.50). Higher maximum impact factor was associated with a shorter time to RPG (Q2 versus lowest: HR 1.28, 95% CI 1.12–1.46; Q3: 1.45, 95% CI 1.24–1.70; Q4: 1.67, 95% CI 1.39–2.02). Mean impact factor and middle-authorships were not associated with time to RPG. Among 687 CDAs with zero associated first- or last-authorships, 158 (23%) achieved later RPG funding. Among those with at least 10 total first- or last-authorships, 1288/1554 (83%) obtained a later RPG. Conclusions A higher number and impact of publications was associated with later independent funding.

Introduction US career development awards (CDAs) are a key step toward independent research careers [1]. These awards provide substantial project and salary support to junior investigators, and provide a pathway to develop the skills and experience to lead large projects. Independent research project grants (RPGs) and individual large project grants (R01 awards) are one of the premier means of supporting biomedical research careers, and are therefore a key outcome of CDAs [2]. These large grants sustain researchers and are a critical source of biomedical innovation in the US [3].
Most CDA recipients will go on to obtain independent research funding, but substantial sex [4][5][6] and race [7][8][9] disparities in this progression exist. Medical school affiliation, institutional funding track record, citations, and caregiving responsibilities partially mediate these disparities [7,10]. Additionally, attracting and retaining a younger and more diverse research workforce had become increasingly difficult, as older researchers obtain a larger share of funding and as the total amount of potential funding has grown more slowly than the research workforce [10,11].
A better understanding of the modifiable characteristics of CDA recipients, such as publication record, that predict later RPG or R01 funding could help provide a blueprint for how CDA recipients might progress to independence.

Materials and methods
I studied US federal CDAs (K01, K08, or K23) with start dates from 2000-2015, excluding any awardees with a prior RPG. The data source was the NIH RePORTER database, 2000-2020.
Potential predictors of research funding included CDA type, funder Institute and Center (IC) for ICs with at least 100 CDAs during the study, whether the CDA required resubmission to achieve funding, change of awardee institution during the CDA period, number of CDAassociated first-author papers, number of CDA-associated middle-author papers, number of CDA-associated last-author papers, highest impact factor of the CDA-associated papers' journals, mean impact factor, awardee institution located in the US, and whether the awardee institution was in the top 20, next 30, or outside the top 50 institutions in number of CDAs. Journal impact factors were drawn from the Web of Science 2019 Journal Citation Reports [12].
I determined the association of each predictor with time to independent research funding. To make this comparison, I constructed Kaplan-Meier survival curves for each type of independent research funding stratified on predictor variables and tested each association using a log rank test. All researchers who did not achieve R01 or RPG funding were censored on December 31, 2020. The independent association of each predictor with later funding was evaluated using a Cox proportional hazard models including all predictors except for funder IC (because there were too many ICs to feasibly include). Finally, to examine the role of publishing and of ICs, the proportion of CDA recipients who went on to independent funding was determined by number of first-and last-authorships, and separately by funder IC.
Time to R01 and time to RPG were each shorter among CDA awardees who changed institution during their CDA, those with higher impact papers, more first, last, or middle authorships associated with the CDA, and among CDAs awarded to top 20 institutions (p<0.0001 for each comparison, Fig 1). There was no association between CDA type (K01, K08, or K23) and time to R01 or RPG.
The multivariable Cox regression model demonstrated that each predictor except middle authorships were independently associated with time to R01 or RPG ( Table 2). For both R01s and RPGs, predictors independently associated with a longer time to independent award were K23, no change of institution during the CDA, lower maximum impact factor of CDA-associated papers, lower mean impact factor of CDA-associated papers, fewer first authorships, fewer last authorships, and affiliation with an institution outside the top 20 CDA-awarded institutions. The strongest predictors were the numbers of first and last authorships.
More CDA-associated first or last authorships substantially improved the proportion of applicants who obtained later R01 or RPG funding (Fig 2). Among those with no first or last authorships, 115/687 (17%) obtained a later R01 and 158/687 (23%) obtained a later RPG. By contrast, of those who had at least 10 total first or last authorships, 1108/1554 (71%) obtained a later R01 and 1288/1554 (83%) obtained a later RPG. ICs varied in later funding for CDA recipients (Fig 3). The National Institute on Aging had the highest proportion of later R01s (195/345, 57%) and RPGs (231/345, 67%). The National Institute of Nursing Research had the fewest later R01s (31/98, 32%) and the National Center for Research Resources had the fewest later RPGs (91/219, 42%).

Discussion
Among 6,744 CDA recipients from 2000-2015, approximately half later obtained US research project funding. The number and impact of publications was a very strong predictor of success in later independent funding: approximately one fifth of those without first or last authorships go on to later funding, whereas approximately four fifths with at least 10 first or last authorships go on to later funding. First and last authorships measure different phenomena [13]. First authorships represent work the author leads, and last authorships represent work the author supervises. Middle authorship frequently reflects contributions to a larger team. Middle authorships were not independently predictive of independent research funding success. Taken together, these findings suggest that CDA recipients with a high number of publications reflecting leadership, mentorship, or both will go on to obtain independent funding. These findings could differ in other nations where funding decisions might rely on other criteria.  This study could not evaluate the reasons that more publications are associated with a shorter time to independent research funding. However, regardless of the actual causes, the implications are clear and reflect the adage "publish or perish." CDA recipients with few publications are unlikely to obtain US federal funding, while those with many are highly likely to do so. As a result, known disparities in early research experience, topic choice, and firstand last-authorships by race [14,15] and gender [16] may, at least in part, mediate funding disparities [8] by those characteristics. Whether a grant applicant's number of publications is a useful barometer of future research impact is unclear. Highly influential scientists may not be funded when publication count is a key metric [17]. Alternative approaches include funding all applicants, funding applicants randomly, automated approaches, rewarding citizenship, or alternative rubrics for evaluation of promising science [18]. However, any single approach has positive and negative features. Absent major changes in the approach to evaluating research grant applications, authorship is likely to continue to be a meaningful contributor to application success. Institutions and leaders seeking to improve local funding rates and narrow disparities in funding should therefore encourage, foster, and monitor leadership and supervision among junior researchers, particularly those funded by CDAs.
Our study has several limitations. First, our study could not account for changes in funding rates and priorities over the 16 years I analyzed. Second, I could not account for CDA recipients who left the workforce prior to the end of the study period, who would therefore have been ineligible for the outcome of RPG funding. I believe workforce departures would be uncommon since most CDA awardees are in their early career. Finally, I could not directly measure the degree of mediation of publications with disparities, as I did not have access to CDA recipient demographics.

Conclusions
Among CDA awardees from 2000-2015, first-and last-authorship and impactful publications were strong independent predictors of a shorter time to R01 or RPG funding. Fostering local research productivity could improve independent grant funding success rates among junior researchers. The effect of improving publication counts on disparities in research funding should be undertaken.